Separable representations for cocktail party processing

Perceptual parsing of a complex acoustic scene requires the following ingredients: (a) a representation that is “separable” in the sense that it allows patterns to be split into parts that belong to diverse sources, (b) rules and cues to guide the partition, and (c) processes that can make sense of the partitioned information in the event that the partition was imperfect and the partitioned information is incomplete. Acoustic waveforms can usefully be represented in the time domain, in the frequency domain, in the cepstral domain, etc. These domains are transforms one of the other, and for tasks such as recognizing speech in quiet they may in principle be used interchangeably. In the presence of noise or competing sources, however, partitioning may be easier within one representation than within others. This paper explores the idea that the auditory system forms a set of diverse representations that are redundant in quiet, but useful in cluttered acoustic scenes. These representations are produced by a combination of spectral analysis in the cochlea and time-domain neural processing within the nervous system. Partitioning is based on cues and rules such as described by Bregman (1990). The final pattern-matching stage must be able to use the possibly incomplete information that survives the partitioning stage. Similar ideas may be applied to machine analysis of complex acoustic scenes.